"What went well"? "What could have been done better"?

http://pdm.medicine.wisc.edu Prehospital and Disaster Medicine Evaluations of the processes involved in an intervention are similar to the way in which diagnoses and treatments are defined in clinical medicine. Diagnoses and interventions cannot be accomplished without knowing the physiology/pathophysiology of the organ system(s) involved. Therapy is directed at the pathophysiology. As noted in the most recent Editor’s Corner, for disaster evaluation purposes, the Logic Model has particular relevance.2 In this model, the current existing status of an individual or a population-at-risk or affected by an event is transformed by a process, into a new, hopefully improved, status. This transformation of a baseline state is accomplished through a process that includes the intervention(s)/response(s). If an intervention meets the objective(s) for which it was intended, but the process used to reach the objective consumed an extra-ordinary amount of resources, should it be used again in the same way in another similar situation, or should the process be examined for critical points of ineff iciency that can be corrected before the intervention is used again. Such an evaluation of the intervention may lead to changes that will make the process more efficient the next time it is employed—the next time the intervention is implemented, it could consume less resources than when it last was implemented. It only is through such analyses that we are able to really learn from what we have learned. If an intervention met the objectives for which it was intended, but also produced effects/impacts that were negative and outweighed the benefits derived from the intervention, should the intervention be repeated in another situation? Should the process be abandoned? Should it be modified in an effort to attenuate the negative effects/impacts? By identifying where in the process the negative effects/impacts were generated, we can modify the process to eliminate those points in the process that generated the resultant negative effects. Thus, we could achieve the benefits without the negative effects, and we truly would have learned from what we have learned. Similarly, when the outcomes are positive and the costs are reasonable, it is valuable to know why the process worked well, to identify the critical points of success, and hopefully, repeat the intervention the next time to benefit the persons/populations affected. When such successes are documented, the process employed should be codified and widely distributed in order that others can apply the same intervention and avoid future pitfalls. Thus, in the conduct of evaluations, it is essential that a dissection of the process used for the intervention(s) be studied. Simply defining the outcome and labeling the project as a “success” contribute little to our learning, and hence, we cannot incorporate what we have learned into future projects. Creative thinking may simply mean the realization that there is no particular virtue in doing things the way we have always done them. Rudolph Flesch